Renewal Processes with Costs and Rewards

We review the theory of renewal reward processes, which describes renewal processes that have some cost or reward associated with each cycle. We present a new simplified proof of the renewal reward theorem that mimics the proof of the elementary renewal theorem and avoids the technicalities in the proof that is presented in most textbooks. Moreover, we mention briefly the extension of the theory to partial rewards, where it is assumed that rewards are not accrued only at renewal epochs but also during the renewal cycle. For this case, we present a counterexample which indicates that the standard conditions for the renewal reward theorem are not sufficient; additional regularity assumptions are necessary. We present a few examples to indicate the usefulness of this theory, where we prove the inspection paradox and Little’s law through the renewal reward theorem. Basic notions and results Many applications of the renewal theory involve rewards or costs (which can be simply seen as a negative reward). For example, consider the classical example of a renewal process: a machine component gets replaced upon failure or upon having operated for T time units. Then the time the n-th component is in service is given by Yn = min{Xn,T }, where Xn is the life of the component. In this example, one might be interested in the rate of the number of replacements in the long run. An extension of this basic setup is as follows. A component that has failed will be replaced at a cost c f , while a component that is replaced while still being operational (and thus at time T ) costs only c < c f . In this case, one might be interested in choosing the optimal time T that minimises the long-run operational costs. The solution to this problem involves the analysis of renewal processes with costs and rewards. Motivated by the above, let {N(t), t > 0} be a renewal process with interarrival times Xn, n > 1, and denote the time of the n-th renewal by S n = X1 + · · · + Xn. Now suppose that at the time of each renewal a reward is received; we denote by Rn the reward received at the end of the n-th cycle. We further assume that (Rn, Xn) is a sequence of i.i.d. random variables, which allows for Rn to depend on Xn. For example, N(t) might count the number of rides a taxi gets up to time t. In this case, Xn is the length of each trip and one reasonably expects the fare Rn to depend on Xn. As usual, we denote by (R, X) the generic bivariate random variable that are distributed identically to the sequence of rewards and interarrival times (Rn, Xn). In the analysis, together with the standard assumption in renewal processes that the interarrival times have a finite expectation E[X] = τ, we will further assume that E[ |R| ] < ∞. The cumulative reward up to time t is given by R(t) = ∑N(t) n=1 Rn, where the sum is taken to be equal to zero in the event that N(t) = 0. Depending ∗Dept. of Mathematics & Computer Science, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands, [email protected]